Skip to content

Conversation

sdangol
Copy link
Contributor

@sdangol sdangol commented Sep 3, 2025

Summary

This PR adds a helper function which decodes a base64 encoded payload and parses the decoded payload with the schema passed to the function

Changes

Please provide a summary of what's being changed

  • Added the helper function to decompress, decode and parse a base64 encoded payload
  • Added tests for the helper function

Please add the issue number below, if no issue is present the PR might get blocked and not be reviewed

Issue number: closes #4412


By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.

@pull-request-size pull-request-size bot added the size/L PRs between 100-499 LOC label Sep 3, 2025
@sdangol sdangol self-assigned this Sep 3, 2025
@boring-cyborg boring-cyborg bot added parser This item relates to the Parser Utility tests PRs that add or change tests labels Sep 3, 2025
@sdangol sdangol requested review from dreamorosi and svozza September 3, 2025 17:38
Copy link
Contributor

@dreamorosi dreamorosi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we use the new helper in the Kinsis schema here (packages/parser/src/schemas/kinesis.ts) similar to how we use the DynamoDB one in the DynamoDB schema here?

@sdangol
Copy link
Contributor Author

sdangol commented Sep 4, 2025

@dreamorosi The dynamodb helper doesn't seem to be used in that schema, it's only referenced in the comments. It's using unmarshallDynamoDB. We would need an external schema to use that helper, right? Can we use it in the kinesis schema?

@dreamorosi
Copy link
Contributor

If I understand the new helper correctly, you should be able to replace these lines here with something like this:

const KinesisDataStreamRecordPayload = z.object({
  kinesisSchemaVersion: z.string(),
  partitionKey: z.string(),
  sequenceNumber: z.string(),
  approximateArrivalTimestamp: z.number(),
  data: Base64Encoded(z.any()), // or z.unknown()
});

and also remove the decompress function and decoder references plus their imports.

@sdangol sdangol requested a review from dreamorosi September 4, 2025 09:01
Copy link
Contributor

@dreamorosi dreamorosi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this

Copy link

sonarqubecloud bot commented Sep 4, 2025

@sdangol sdangol merged commit 1554360 into main Sep 4, 2025
37 checks passed
@sdangol sdangol deleted the feat/parser-decompress-helper branch September 4, 2025 10:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

parser This item relates to the Parser Utility size/L PRs between 100-499 LOC tests PRs that add or change tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature request: Provide a helper function to decode a base64 encoded payload

2 participants